feat(server): New TTL system, enforce max queue length limits, lazy waitpoint creation #2980
Conversation
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughCentralizes queue-size logic (new v3/queueLimits utility and environment queueSizeLimit exposure) and adds an LRU cache for environment queue lengths. Refactors queue validation to per-queue semantics (resolveQueueNamesForBatchItems, validateMultipleQueueLimits) and surfaces itemsSkipped/runCount through batch streaming APIs. Introduces per-item retry for batch queue processing, batch-run-count updates, and a TriggerFailedTaskService for creating pre-failed runs. Adds a TTL expiration subsystem (batched TTL consumers, Redis TTL scripts, ttlSystem callback) and lazy get-or-create waitpoints with related waitpoint APIs. Numerous RunEngine/RunQueue/BatchQueue public API additions and tests updated; UI presenters and routes updated to use the single queueSize quota. Estimated code review effort🎯 5 (Critical) | ⏱️ ~180 minutes 🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Review CompleteYour review story is ready! Comment !reviewfast on this PR to re-generate the story. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@apps/webapp/app/presenters/v3/LimitsPresenter.server.ts`:
- Around line 170-194: The code silently treats a missing runtime environment as
a zero queue by checking runtimeEnv and continuing; change this to explicitly
handle the case where runtimeEnv is null for the required environmentId by
returning/throwing an error (or returning a clear "not found" result) instead of
falling through to the deployed default logic—locate the block where runtimeEnv
is fetched (the runtimeEnvironment.findFirst call and the subsequent use of
runtimeEnv, currentQueueSize and engine.lengthOfEnvQueue) and add an explicit
branch that raises a descriptive error or returns an appropriate 404-style
response when runtimeEnv is falsy so callers don’t get misleading limits
computed for a non-existent environment.
🧹 Nitpick comments (1)
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts (1)
311-329: Queue size limit logic is duplicated withEnvironmentQueuePresenter.The limit computation logic (lines 314-317) and source determination (lines 321-328) duplicate the same pattern found in
EnvironmentQueuePresenter.server.ts(lines 44-47). Consider extracting this into a shared utility function to ensure consistency and reduce maintenance burden.♻️ Example shared utility
// In a shared utils file, e.g., ~/utils/queueSizeLimit.ts export function getQueueSizeLimitForEnvironment( environmentType: string, organization: { maximumDevQueueSize: number | null; maximumDeployedQueueSize: number | null }, defaultDevLimit: number | null, defaultDeployedLimit: number | null ): { limit: number | null; source: "plan" | "default" } { if (environmentType === "DEVELOPMENT") { return { limit: organization.maximumDevQueueSize ?? defaultDevLimit ?? null, source: organization.maximumDevQueueSize ? "plan" : "default", }; } return { limit: organization.maximumDeployedQueueSize ?? defaultDeployedLimit ?? null, source: organization.maximumDeployedQueueSize ? "plan" : "default", }; }
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx
🧰 Additional context used
📓 Path-based instructions (7)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
**/*.{ts,tsx}: Always import tasks from@trigger.dev/sdk, never use@trigger.dev/sdk/v3or deprecatedclient.defineJobpattern
Every Trigger.dev task must be exported and have a uniqueidproperty with no timeouts in the run function
Files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Import from
@trigger.dev/coreusing subpaths only, never import from root
Files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
apps/webapp/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access all environment variables through the
envexport ofenv.server.tsinstead of directly accessingprocess.envin the Trigger.dev webapp
Files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: When importing from@trigger.dev/corein the webapp, use subpath exports from the package.json instead of importing from the root path
Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webappAccess environment variables via
envexport fromapps/webapp/app/env.server.ts, never useprocess.envdirectly
Files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
**/*.{js,ts,jsx,tsx,json,md,yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier before committing
Files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
🧠 Learnings (10)
📚 Learning: 2025-12-08T15:19:56.823Z
Learnt from: 0ski
Repo: triggerdotdev/trigger.dev PR: 2760
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx:278-281
Timestamp: 2025-12-08T15:19:56.823Z
Learning: In apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx, the tableState search parameter uses intentional double-encoding: the parameter value contains a URL-encoded URLSearchParams string, so decodeURIComponent(value("tableState") ?? "") is required to fully decode it before parsing with new URLSearchParams(). This pattern allows bundling multiple filter/pagination params as a single search parameter.
Applied to files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
Applied to files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/app/v3/presenters/**/*.server.{ts,tsx} : Organize presenters in the webapp following the pattern `app/v3/presenters/*/*.server.ts` to move complex loader code into classes
Applied to files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx
📚 Learning: 2026-01-15T11:50:06.067Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.067Z
Learning: Applies to apps/webapp/**/*.{ts,tsx} : Access environment variables via `env` export from `apps/webapp/app/env.server.ts`, never use `process.env` directly
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/app/**/*.{ts,tsx} : Access all environment variables through the `env` export of `env.server.ts` instead of directly accessing `process.env` in the Trigger.dev webapp
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts
📚 Learning: 2026-01-15T11:50:06.067Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.067Z
Learning: Applies to apps/webapp/**/*.test.{ts,tsx} : For testable code in the webapp, never import env.server.ts in test files - pass configuration as options instead
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/**/*.test.{ts,tsx} : Test files should only import classes and functions from `app/**/*.ts` files and should not import `env.server.ts` directly or indirectly; pass configuration through options instead
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts
📚 Learning: 2025-11-14T16:03:06.917Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2681
File: apps/webapp/app/services/platform.v3.server.ts:258-302
Timestamp: 2025-11-14T16:03:06.917Z
Learning: In `apps/webapp/app/services/platform.v3.server.ts`, the `getDefaultEnvironmentConcurrencyLimit` function intentionally throws an error (rather than falling back to org.maximumConcurrencyLimit) when the billing client returns undefined plan limits. This fail-fast behavior prevents users from receiving more concurrency than their plan entitles them to. The org.maximumConcurrencyLimit fallback is only for self-hosted deployments where no billing client exists.
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2026-01-12T17:18:09.451Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2870
File: apps/webapp/app/services/redisConcurrencyLimiter.server.ts:56-66
Timestamp: 2026-01-12T17:18:09.451Z
Learning: In `apps/webapp/app/services/redisConcurrencyLimiter.server.ts`, the query concurrency limiter will not be deployed with Redis Cluster mode, so multi-key operations (keyKey and globalKey in different hash slots) are acceptable and will function correctly in standalone Redis mode.
Applied to files:
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Use the Run Engine 2.0 from `internal/run-engine` for new run lifecycle code in the webapp instead of the legacy run engine
Applied to files:
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts
🧬 Code graph analysis (2)
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx (2)
apps/webapp/app/utils/numberFormatter.ts (1)
formatNumberCompact(3-5)apps/webapp/app/components/primitives/Tooltip.tsx (1)
InfoIconTooltip(110-139)
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts (1)
apps/webapp/app/v3/runEngine.server.ts (1)
engine(11-11)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: sdk-compat / Cloudflare Workers
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: sdk-compat / Deno Runtime
- GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
- GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
- GitHub Check: sdk-compat / Bun Runtime
- GitHub Check: typecheck / typecheck
🔇 Additional comments (5)
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts (1)
1-56: LGTM!The implementation correctly extends the Environment type with
queueSizeLimitand computes it based on environment type with appropriate fallback chain (org override → env var → null). Environment variable access follows coding guidelines via~/env.server.apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx (1)
510-511: LGTM!Clean consolidation from separate dev/deployed queue size quotas to a unified
queueSizequota, conditionally rendered only when a limit is set. This aligns well with theLimitsPresenterchanges.apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx (2)
349-391: LGTM!The queue size limit display with color-coded usage indicators is well-implemented. The conditional rendering shows the limit suffix when
queueSizeLimitexists, with appropriate fallback to the paused indicator.Minor note: The component has
envfromuseEnvironment()(line 313) shadowing theenvimport from~/env.server(line 57). This works correctly since the loader uses the importedenvand the component uses the hook result, but renaming one variable (e.g.,runtimeEnvfor the hook result) could improve clarity.
1145-1152: LGTM!Clean helper function with consistent threshold logic (90% warning, 100% error). Returning
undefinedwhen no limit is set allows proper fallback behavior in the calling code.apps/webapp/app/presenters/v3/LimitsPresenter.server.ts (1)
15-15: LGTM on the structural changes.The import of
engineand the updatedLimitsResulttype with consolidatedqueueSize: QuotaInfoproperly support the new unified queue size quota feature.Also applies to: 70-70
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@apps/webapp/app/env.server.ts`:
- Around line 536-539: The MAXIMUM_DEV_QUEUE_SIZE default of 500 introduces an
unintended hard cap for dev environments because guardQueueSizeLimitsForEnv
reads this value and enforces it; remove the .default(500) so
MAXIMUM_DEV_QUEUE_SIZE remains optional/undefined unless explicitly set in the
environment, leaving the z.coerce.number().int().optional() schema for
MAXIMUM_DEV_QUEUE_SIZE and ensure any code calling guardQueueSizeLimitsForEnv
continues to treat undefined as "no cap" (verify guardQueueSizeLimitsForEnv
behavior and update it only if it currently treats undefined incorrectly).
In `@apps/webapp/app/presenters/v3/LimitsPresenter.server.ts`:
- Around line 313-319: The queue-size quota object (queueSize) in
LimitsPresenter.server.ts is missing the isUpgradable flag so the UI never shows
an Upgrade action; update the queueSize payload returned by the presenter to
include isUpgradable: true for plan-based limits (and add canExceed: true|false
as appropriate), e.g., set isUpgradable to true when
getQueueSizeLimitSource(environmentType, organization) indicates a plan-based
source and ensure the UI-facing fields (queueSize.currentUsage, queueSize.limit,
queueSize.source) remain unchanged.
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
apps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx
🧰 Additional context used
📓 Path-based instructions (7)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
**/*.{ts,tsx}: Always import tasks from@trigger.dev/sdk, never use@trigger.dev/sdk/v3or deprecatedclient.defineJobpattern
Every Trigger.dev task must be exported and have a uniqueidproperty with no timeouts in the run function
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Import from
@trigger.dev/coreusing subpaths only, never import from root
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
apps/webapp/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access all environment variables through the
envexport ofenv.server.tsinstead of directly accessingprocess.envin the Trigger.dev webapp
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: When importing from@trigger.dev/corein the webapp, use subpath exports from the package.json instead of importing from the root path
Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webappAccess environment variables via
envexport fromapps/webapp/app/env.server.ts, never useprocess.envdirectly
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
**/*.{js,ts,jsx,tsx,json,md,yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier before committing
Files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
🧠 Learnings (8)
📓 Common learnings
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/v3/utils/queueLimits.server.ts
📚 Learning: 2025-11-14T16:03:06.917Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2681
File: apps/webapp/app/services/platform.v3.server.ts:258-302
Timestamp: 2025-11-14T16:03:06.917Z
Learning: In `apps/webapp/app/services/platform.v3.server.ts`, the `getDefaultEnvironmentConcurrencyLimit` function intentionally throws an error (rather than falling back to org.maximumConcurrencyLimit) when the billing client returns undefined plan limits. This fail-fast behavior prevents users from receiving more concurrency than their plan entitles them to. The org.maximumConcurrencyLimit fallback is only for self-hosted deployments where no billing client exists.
Applied to files:
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-12-08T15:19:56.823Z
Learnt from: 0ski
Repo: triggerdotdev/trigger.dev PR: 2760
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx:278-281
Timestamp: 2025-12-08T15:19:56.823Z
Learning: In apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx, the tableState search parameter uses intentional double-encoding: the parameter value contains a URL-encoded URLSearchParams string, so decodeURIComponent(value("tableState") ?? "") is required to fully decode it before parsing with new URLSearchParams(). This pattern allows bundling multiple filter/pagination params as a single search parameter.
Applied to files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use the `task()` function from `trigger.dev/sdk/v3` to define tasks with id and run properties
Applied to files:
apps/webapp/app/runEngine/concerns/queues.server.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/app/**/*.{ts,tsx} : Access all environment variables through the `env` export of `env.server.ts` instead of directly accessing `process.env` in the Trigger.dev webapp
Applied to files:
apps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Specify runtime environment (node or bun) in trigger.config.ts using the `runtime` property
Applied to files:
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2026-01-12T17:18:09.451Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2870
File: apps/webapp/app/services/redisConcurrencyLimiter.server.ts:56-66
Timestamp: 2026-01-12T17:18:09.451Z
Learning: In `apps/webapp/app/services/redisConcurrencyLimiter.server.ts`, the query concurrency limiter will not be deployed with Redis Cluster mode, so multi-key operations (keyKey and globalKey in different hash slots) are acceptable and will function correctly in standalone Redis mode.
Applied to files:
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts
🧬 Code graph analysis (2)
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts (1)
apps/webapp/app/v3/utils/queueLimits.server.ts (1)
getQueueSizeLimit(24-33)
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts (2)
apps/webapp/app/v3/runEngine.server.ts (1)
engine(11-11)apps/webapp/app/v3/utils/queueLimits.server.ts (2)
getQueueSizeLimit(24-33)getQueueSizeLimitSource(42-51)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (26)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: sdk-compat / Cloudflare Workers
- GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
- GitHub Check: sdk-compat / Bun Runtime
- GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
- GitHub Check: sdk-compat / Deno Runtime
- GitHub Check: typecheck / typecheck
🔇 Additional comments (7)
apps/webapp/app/runEngine/concerns/queues.server.ts (3)
18-33: LRU cache setup looks solid.The singleton + bounded LRU store should help reduce Redis churn without unbounded memory growth.
298-317: Confirm stale cache is acceptable for enforcement.
guardQueueSizeLimitsForEnvnow relies on an SWR cache; with a ~30s TTL, queue size can be undercounted and allow bursts above the limit. If this is a hard/billing limit, consider bypassing the cache for enforcement (or forcing a fresh read when nearing the limit).
68-68: No issues to flag in these edits.Also applies to: 94-96, 116-118, 130-131
apps/webapp/app/v3/utils/queueLimits.server.ts (1)
7-51: Queue-limit helpers look good.Clear resolution order and source labeling; the API surface is minimal and consistent.
apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts (1)
4-52:queueSizeLimitwiring looks consistent.The new field is derived in one place and keeps the presenter output cohesive.
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts (1)
2-18: API/type expansion for queue limits looks consistent.The new environmentType parameter and consolidated quota surface align with the updated limit logic.
Also applies to: 58-96
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx (1)
81-86: Loader and quota table updates look good.Passing
environmentTypeand gating the queue-size quota on a defined limit is consistent with the new presenter shape.Also applies to: 511-513
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
ericallam
force-pushed
the
ea-branch-117
branch
from
a50a5f5 to
aaea8d6
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
internal-packages/run-engine/src/engine/tests/ttl.test.ts (1)
1-9:⚠️ Potential issue | 🟡 MinorMissing
viimport from vitest.Similar to lazyWaitpoint.test.ts, the
viglobal is used but not imported.🔧 Proposed fix
import { containerTest, assertNonNullable } from "@internal/testcontainers"; import { trace } from "@internal/tracing"; -import { expect } from "vitest"; +import { expect, vi } from "vitest"; import { RunEngine } from "../index.js";apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx (1)
5-5:⚠️ Potential issue | 🟡 MinorUse subpath export for
tryCatchfrom@trigger.dev/core/utils.Per coding guidelines, imports from
@trigger.dev/corein the webapp must use subpath exports from the package.json. Changeimport { tryCatch } from "@trigger.dev/core";toimport { tryCatch } from "@trigger.dev/core/utils";.
🤖 Fix all issues with AI agents
In `@internal-packages/run-engine/src/batch-queue/completionTracker.ts`:
- Around line 112-130: The updateRunCount method currently trusts newRunCount;
add input validation in updateRunCount (in completionTracker.ts) to reject
negative, NaN, non-finite or non-integer values before calling
getMeta/storeMeta: use Number.isInteger(newRunCount) && newRunCount >= 0 (and
optionally Number.isFinite) and if the check fails log an error via
this.logger.error with batchId and the invalid value and then throw an Error or
return a rejected Promise so the invalid value is never persisted; keep existing
behavior for meta retrieval (getMeta) and storage (storeMeta) otherwise.
In `@internal-packages/run-engine/src/batch-queue/index.ts`:
- Around line 380-386: The updateRunCount method can lower runCount below the
already-processed count and leave a batch stuck; before calling
this.completionTracker.updateRunCount(batchId, newRunCount) clamp newRunCount to
at least the current processed count (e.g. newRunCount = Math.max(newRunCount,
this.completionTracker.getProcessedCount(batchId))) or, alternatively, after
updating call into the completion tracker to run the completion/finalization
check (e.g. this.completionTracker.checkCompletion/finalizeBatch for batchId) so
that if processedCount >= runCount the batch is finalized immediately; update
updateRunCount to perform one of these fixes using the existing
completionTracker APIs.
In `@internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.ts`:
- Around line 1-8: The test uses the vitest runner's mock/time API via
vi.setConfig but never imports vi; update the imports to explicitly import vi
from 'vitest' (for example change the existing import of expect to import {
expect, vi } from "vitest") so the vi symbol used by vi.setConfig is defined and
the test runner APIs are available.
In `@internal-packages/run-engine/src/run-queue/index.ts`:
- Around line 1238-1304: The code currently calls `#expireTtlRuns`(shard, now,
batchSize) which deletes/marks runs as expired before invoking
options.ttlSystem!.callback(expiredRuns), so callback failures only get logged
and those runs can be lost; update `#startTtlConsumer` (or `#expireTtlRuns`) so that
expiration is durable: either (A) defer permanent deletion/marking of runs until
after the callback completes successfully (move the delete/mark logic to after
options.ttlSystem!.callback returns) or (B) if deletion must happen first, add a
durable retry/enqueue path when the callback throws — e.g., push failed expired
run IDs to a retry store (Redis list/sorted set) with metadata and backoff, and
spawn a retry worker to reprocess them; ensure the callback and retry path are
idempotent and update logging to include run IDs and shard when queuing retries.
- Around line 2561-2637: expireTtlRuns currently removes run entries from
per-queue structures but never updates the master queue (masterQueueKey),
leaving stale entries; update the logic so that either (A) the Lua script also
computes the corresponding master/shard key from queueKey and ZREM the runId
from masterQueueKey (or adjust scores) for each expired member, or (B) return
the affected queueKey values (not just member) from expireTtlRuns so the
TypeScript caller can call the existing rebalance/cleanup routine for
masterQueueKey per returned queue; locate expireTtlRuns, the queueKey local
variable and the final results/return section to implement one of these fixes.
🧹 Nitpick comments (7)
packages/core/src/v3/schemas/api.ts (1)
412-424: Tighten validation for new count fields.
These are counts, so restricting them to non‑negative integers prevents accidental floats/negatives in API responses.♻️ Suggested update
- itemsSkipped: z.number().optional(), + itemsSkipped: z.number().int().nonnegative().optional(), ... - runCount: z.number().optional(), + runCount: z.number().int().nonnegative().optional(),apps/webapp/app/runEngine/concerns/queues.server.ts (1)
421-432: Consider potential cache stampede scenario.The SWR pattern handles stale reads well, but when the cache is completely cold (e.g., after deployment), multiple concurrent requests for the same queue could all trigger Redis lookups simultaneously before the first one populates the cache.
For most use cases this is acceptable, but if queue size lookups become a bottleneck, consider adding a coalescing mechanism.
apps/webapp/app/runEngine/services/triggerTask.server.ts (1)
258-269: Queue validation correctly uses resolved queue name.The reordering to validate after
resolveQueuePropertiesis correct since the queue name is needed for per-queue validation.Consider enhancing the error message to include the queue name for better debuggability:
💡 Optional: Include queue name in error message
if (!queueSizeGuard.ok) { throw new ServiceValidationError( - `Cannot trigger ${taskId} as the queue size limit for this environment has been reached. The maximum size is ${queueSizeGuard.maximumSize}` + `Cannot trigger ${taskId} as the queue size limit for queue '${queueName}' has been reached. The maximum size is ${queueSizeGuard.maximumSize}` ); }internal-packages/run-engine/src/run-queue/types.ts (1)
56-128: ConvertRunQueueKeyProducerinterface to a type alias to align with TypeScript guidelines.♻️ Suggested refactor
-export interface RunQueueKeyProducer { +export type RunQueueKeyProducer = { //queue queueKey( orgId: string, projId: string, envId: string, queue: string, concurrencyKey?: string ): string; queueKey(env: RunQueueKeyProducerEnvironment, queue: string, concurrencyKey?: string): string; legacyMasterQueueKey(masterQueueName: string): string; masterQueueKeyForEnvironment(envId: string, shardCount: number): string; masterQueueKeyForShard(shard: number): string; masterQueueShardForEnvironment(envId: string, shardCount: number): number; workerQueueKey(workerQueue: string): string; envQueueKey(env: RunQueueKeyProducerEnvironment): string; envQueueKeyFromQueue(queue: string): string; queueConcurrencyLimitKey(env: RunQueueKeyProducerEnvironment, queue: string): string; queueConcurrencyLimitKeyFromQueue(queue: string): string; queueCurrentConcurrencyKeyFromQueue(queue: string): string; queueCurrentConcurrencyKey( env: RunQueueKeyProducerEnvironment, queue: string, concurrencyKey?: string ): string; queueCurrentDequeuedKeyFromQueue(queue: string): string; queueCurrentDequeuedKey( env: RunQueueKeyProducerEnvironment, queue: string, concurrencyKey?: string ): string; //env oncurrency envCurrentConcurrencyKey(env: EnvDescriptor): string; envCurrentConcurrencyKey(env: RunQueueKeyProducerEnvironment): string; envConcurrencyLimitKey(env: EnvDescriptor): string; envConcurrencyLimitKey(env: RunQueueKeyProducerEnvironment): string; envCurrentDequeuedKey(env: EnvDescriptor): string; envCurrentDequeuedKey(env: RunQueueKeyProducerEnvironment): string; envConcurrencyLimitBurstFactorKey(env: EnvDescriptor): string; envConcurrencyLimitBurstFactorKey(env: RunQueueKeyProducerEnvironment): string; envConcurrencyLimitBurstFactorKeyFromQueue(queue: string): string; envConcurrencyLimitKeyFromQueue(queue: string): string; envCurrentConcurrencyKeyFromQueue(queue: string): string; envCurrentDequeuedKeyFromQueue(queue: string): string; //message payload messageKeyPrefixFromQueue(queue: string): string; messageKey(orgId: string, messageId: string): string; //utils orgIdFromQueue(queue: string): string; envIdFromQueue(queue: string): string; projectIdFromQueue(queue: string): string; descriptorFromQueue(queue: string): QueueDescriptor; deadLetterQueueKey(env: RunQueueKeyProducerEnvironment): string; deadLetterQueueKey(env: EnvDescriptor): string; deadLetterQueueKeyFromQueue(queue: string): string; // Concurrency sweeper methods markedForAckKey(): string; currentConcurrencySetKeyScanPattern(): string; // TTL system methods ttlQueueKeyForShard(shard: number): string; -} +};internal-packages/run-engine/src/engine/systems/ttlSystem.ts (2)
252-260: Unnecessary type cast for TaskRunStatus.The string literal
"EXPIRED"is a validTaskRunStatusvalue, so theas TaskRunStatuscast is redundant.🧹 Minor cleanup
await this.$.prisma.taskRun.updateMany({ where: { id: { in: runIdsToExpire } }, data: { - status: "EXPIRED" as TaskRunStatus, + status: "EXPIRED", completedAt: now, expiredAt: now, }, });
263-321: Consider error handling for individual run processing within the batch.If any single run fails during snapshot creation or waitpoint completion,
Promise.allwill reject and subsequent runs won't be processed. Since the batchupdateManyon line 252 has already succeeded, this could leave some runs in a partially processed state (status=EXPIRED but no snapshot).For improved resilience, consider using
Promise.allSettledand collecting failures:💡 Suggested approach
- await Promise.all( - runsToExpire.map(async (run) => { + const results = await Promise.allSettled( + runsToExpire.map(async (run) => { // ... existing processing ... expired.push(run.id); + return run.id; }) ); + + // Log any failures but don't throw - the status update already succeeded + for (const result of results) { + if (result.status === "rejected") { + this.$.logger.error("Failed to fully process expired run", { + error: result.reason, + }); + } + }internal-packages/run-engine/src/run-queue/index.ts (1)
108-110: Use a type alias for the callback.This project prefers
typeoverinterfacein TS; a function-shaped callback is a clean fit for a type alias.♻️ Suggested change
-export interface TtlSystemCallback { - (runs: Array<{ queueKey: string; runId: string; orgId: string }>): Promise<void>; -} +export type TtlSystemCallback = ( + runs: Array<{ queueKey: string; runId: string; orgId: string }> +) => Promise<void>;As per coding guidelines: Use types over interfaces for TypeScript.
📜 Review details
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
references/hello-world/src/trigger/batches.tsis excluded by!references/**
📒 Files selected for processing (32)
apps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsxapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsapps/webapp/app/runEngine/services/createBatch.server.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsinternal-packages/run-engine/src/batch-queue/completionTracker.tsinternal-packages/run-engine/src/batch-queue/index.tsinternal-packages/run-engine/src/engine/index.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsinternal-packages/run-engine/src/engine/systems/waitpointSystem.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.tsinternal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsinternal-packages/run-engine/src/run-queue/types.tspackages/core/src/v3/schemas/api.tspackages/trigger-sdk/src/v3/shared.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx
- apps/webapp/app/presenters/v3/EnvironmentQueuePresenter.server.ts
🧰 Additional context used
📓 Path-based instructions (12)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Use types over interfaces for TypeScript
Avoid using enums; prefer string unions or const objects instead
**/*.{ts,tsx}: Always import tasks from@trigger.dev/sdk, never use@trigger.dev/sdk/v3or deprecatedclient.defineJobpattern
Every Trigger.dev task must be exported and have a uniqueidproperty with no timeouts in the run function
Files:
packages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/createBatch.server.tsinternal-packages/run-engine/src/run-queue/types.tsinternal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/v3/utils/queueLimits.server.tsinternal-packages/run-engine/src/engine/systems/waitpointSystem.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsapps/webapp/app/v3/runEngine.server.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsapps/webapp/app/env.server.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsinternal-packages/run-engine/src/batch-queue/index.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsapps/webapp/app/runEngine/types.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsapps/webapp/app/runEngine/concerns/queues.server.tsinternal-packages/run-engine/src/run-queue/index.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxinternal-packages/run-engine/src/engine/index.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use zod for validation in packages/core and apps/webapp
Files:
packages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/createBatch.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use function declarations instead of default exports
Import from
@trigger.dev/coreusing subpaths only, never import from root
Files:
packages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/createBatch.server.tsinternal-packages/run-engine/src/run-queue/types.tsinternal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/v3/utils/queueLimits.server.tsinternal-packages/run-engine/src/engine/systems/waitpointSystem.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsapps/webapp/app/v3/runEngine.server.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsapps/webapp/app/env.server.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsinternal-packages/run-engine/src/batch-queue/index.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsapps/webapp/app/runEngine/types.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsapps/webapp/app/runEngine/concerns/queues.server.tsinternal-packages/run-engine/src/run-queue/index.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxinternal-packages/run-engine/src/engine/index.ts
**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)
**/*.ts: When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs
Do not use high-cardinality attributes in OTEL metrics such as UUIDs/IDs (envId, userId, runId, projectId, organizationId), unbounded integers (itemCount, batchSize, retryCount), timestamps (createdAt, startTime), or free-form strings (errorMessage, taskName, queueName)
When exporting OTEL metrics via OTLP to Prometheus, be aware that the exporter automatically adds unit suffixes to metric names (e.g., 'my_duration_ms' becomes 'my_duration_ms_milliseconds', 'my_counter' becomes 'my_counter_total'). Account for these transformations when writing Grafana dashboards or Prometheus queries
Files:
packages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/createBatch.server.tsinternal-packages/run-engine/src/run-queue/types.tsinternal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/v3/utils/queueLimits.server.tsinternal-packages/run-engine/src/engine/systems/waitpointSystem.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsapps/webapp/app/v3/runEngine.server.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsapps/webapp/app/env.server.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsinternal-packages/run-engine/src/batch-queue/index.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsapps/webapp/app/runEngine/types.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsapps/webapp/app/runEngine/concerns/queues.server.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/engine/index.ts
**/*.{js,ts,jsx,tsx,json,md,yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
Format code using Prettier before committing
Files:
packages/core/src/v3/schemas/api.tsapps/webapp/app/runEngine/services/createBatch.server.tsinternal-packages/run-engine/src/run-queue/types.tsinternal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/types.tsapps/webapp/app/v3/utils/queueLimits.server.tsinternal-packages/run-engine/src/engine/systems/waitpointSystem.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsapps/webapp/app/v3/runEngine.server.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsapps/webapp/app/env.server.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsinternal-packages/run-engine/src/batch-queue/index.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsapps/webapp/app/runEngine/types.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsapps/webapp/app/runEngine/concerns/queues.server.tsinternal-packages/run-engine/src/run-queue/index.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsxinternal-packages/run-engine/src/engine/index.ts
{packages,integrations}/**/*
📄 CodeRabbit inference engine (CLAUDE.md)
Add a changeset when modifying any public package in
packages/*orintegrations/*usingpnpm run changeset:add
Files:
packages/core/src/v3/schemas/api.tspackages/trigger-sdk/src/v3/shared.ts
apps/webapp/app/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access all environment variables through the
envexport ofenv.server.tsinstead of directly accessingprocess.envin the Trigger.dev webapp
Files:
apps/webapp/app/runEngine/services/createBatch.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
apps/webapp/**/*.{ts,tsx}: When importing from@trigger.dev/corein the webapp, use subpath exports from the package.json instead of importing from the root path
Follow the Remix 2.1.0 and Express server conventions when updating the main trigger.dev webappAccess environment variables via
envexport fromapps/webapp/app/env.server.ts, never useprocess.envdirectly
Files:
apps/webapp/app/runEngine/services/createBatch.server.tsapps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsapps/webapp/app/v3/runEngine.server.tsapps/webapp/app/env.server.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsapps/webapp/app/runEngine/concerns/queues.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Use vitest for all tests in the Trigger.dev repository
Files:
internal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.ts
**/*.test.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.test.{ts,tsx,js,jsx}: Test files should live beside the files under test and use descriptivedescribeanditblocks
Tests should avoid mocks or stubs and use the helpers from@internal/testcontainerswhen Redis or Postgres are needed
Use vitest for running unit tests
**/*.test.{ts,tsx,js,jsx}: Use vitest exclusively for testing and never mock anything - use testcontainers instead
Place test files next to source files with naming pattern: source file (e.g.,MyService.ts) →MyService.test.ts
Files:
internal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
Use testcontainers helpers (
redisTest,postgresTest,containerTest) from@internal/testcontainersfor Redis/PostgreSQL testing instead of mocks
Files:
internal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.ts
packages/trigger-sdk/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Files:
packages/trigger-sdk/src/v3/shared.ts
🧠 Learnings (40)
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeyTTL` option to define a time window during which duplicate triggers return the original run
Applied to files:
internal-packages/run-engine/src/run-queue/types.tsinternal-packages/run-engine/src/engine/types.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/run-queue/keyProducer.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsinternal-packages/run-engine/src/run-queue/index.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2026-01-12T17:18:09.451Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2870
File: apps/webapp/app/services/redisConcurrencyLimiter.server.ts:56-66
Timestamp: 2026-01-12T17:18:09.451Z
Learning: In `apps/webapp/app/services/redisConcurrencyLimiter.server.ts`, the query concurrency limiter will not be deployed with Redis Cluster mode, so multi-key operations (keyKey and globalKey in different hash slots) are acceptable and will function correctly in standalone Redis mode.
Applied to files:
internal-packages/run-engine/src/run-queue/types.tsapps/webapp/app/v3/runEngine.server.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use metadata methods (set, del, replace, append, remove, increment, decrement, stream, flush) to update metadata during task execution
Applied to files:
internal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsapps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `runs.subscribeToBatch()` to subscribe to changes for all runs in a batch
Applied to files:
internal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/batch-queue/index.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Attach metadata to task runs using the metadata option when triggering, and access/update it inside runs using metadata functions
Applied to files:
internal-packages/run-engine/src/batch-queue/completionTracker.tspackages/trigger-sdk/src/v3/shared.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `.withStreams()` to subscribe to realtime streams from task metadata in addition to run changes
Applied to files:
packages/trigger-sdk/src/v3/shared.tsapps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Subscribe to run updates using `runs.subscribeToRun()` for realtime monitoring of task execution
Applied to files:
packages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.batchTrigger()` to trigger multiple runs of a task from inside another task
Applied to files:
packages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.batchTrigger()` to trigger multiple runs of a single task with different payloads
Applied to files:
packages/trigger-sdk/src/v3/shared.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: Applies to packages/trigger-sdk/**/*.{ts,tsx} : In the Trigger.dev SDK (packages/trigger-sdk), prefer isomorphic code like fetch and ReadableStream instead of Node.js-specific code
Applied to files:
packages/trigger-sdk/src/v3/shared.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.trigger()` to trigger multiple different tasks at once from backend code
Applied to files:
packages/trigger-sdk/src/v3/shared.tsapps/webapp/app/runEngine/services/streamBatchItems.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTaskAndWait()` to batch trigger tasks by passing task instances and wait for results
Applied to files:
packages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerAndWait()` to batch trigger multiple different tasks and wait for results
Applied to files:
packages/trigger-sdk/src/v3/shared.tsinternal-packages/run-engine/src/engine/systems/waitpointSystem.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-14T16:03:06.917Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2681
File: apps/webapp/app/services/platform.v3.server.ts:258-302
Timestamp: 2025-11-14T16:03:06.917Z
Learning: In `apps/webapp/app/services/platform.v3.server.ts`, the `getDefaultEnvironmentConcurrencyLimit` function intentionally throws an error (rather than falling back to org.maximumConcurrencyLimit) when the billing client returns undefined plan limits. This fail-fast behavior prevents users from receiving more concurrency than their plan entitles them to. The org.maximumConcurrencyLimit fallback is only for self-hosted deployments where no billing client exists.
Applied to files:
apps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Control concurrency using the `queue` property with `concurrencyLimit` option
Applied to files:
apps/webapp/app/v3/utils/queueLimits.server.tsapps/webapp/app/runEngine/services/triggerTask.server.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/concerns/queues.server.tsinternal-packages/run-engine/src/run-queue/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use the `task()` function from `trigger.dev/sdk/v3` to define tasks with id and run properties
Applied to files:
internal-packages/run-engine/src/engine/systems/waitpointSystem.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.tsapps/webapp/app/runEngine/concerns/queues.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.triggerAndWait()` to trigger a task and wait for its result from a parent task
Applied to files:
internal-packages/run-engine/src/engine/systems/waitpointSystem.tsinternal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:26:44.496Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/executing-commands.mdc:0-0
Timestamp: 2025-11-27T16:26:44.496Z
Learning: For running tests, navigate into the package directory and run `pnpm run test --run` to enable single-file test execution (e.g., `pnpm run test ./src/engine/tests/ttl.test.ts --run`)
Applied to files:
internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsinternal-packages/run-engine/src/engine/tests/attemptFailures.test.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:26:37.432Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-11-27T16:26:37.432Z
Learning: Applies to **/*.{test,spec}.{ts,tsx} : Use vitest for all tests in the Trigger.dev repository
Applied to files:
internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.ts
📚 Learning: 2026-01-15T11:50:06.067Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-01-15T11:50:06.067Z
Learning: Applies to **/*.{ts,tsx} : Every Trigger.dev task must be exported and have a unique `id` property with no timeouts in the run function
Applied to files:
internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsapps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-10-08T11:48:12.327Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2593
File: packages/core/src/v3/workers/warmStartClient.ts:168-170
Timestamp: 2025-10-08T11:48:12.327Z
Learning: The trigger.dev runners execute only in Node 21 and 22 environments, so modern Node.js APIs like AbortSignal.any (introduced in v20.3.0) are supported.
Applied to files:
internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.ts
📚 Learning: 2025-07-12T18:06:04.133Z
Learnt from: matt-aitken
Repo: triggerdotdev/trigger.dev PR: 2264
File: apps/webapp/app/services/runsRepository.server.ts:172-174
Timestamp: 2025-07-12T18:06:04.133Z
Learning: In apps/webapp/app/services/runsRepository.server.ts, the in-memory status filtering after fetching runs from Prisma is intentionally used as a workaround for ClickHouse data delays. This approach is acceptable because the result set is limited to a maximum of 100 runs due to pagination, making the performance impact negligible.
Applied to files:
internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsinternal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/tests/ttl.test.tsapps/webapp/app/runEngine/concerns/idempotencyKeys.server.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `yourTask.batchTriggerAndWait()` to batch trigger tasks and wait for all results from a parent task
Applied to files:
internal-packages/run-engine/src/engine/tests/lazyWaitpoint.test.tsapps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.tsinternal-packages/run-engine/src/engine/index.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `tasks.trigger()` with type-only imports to trigger tasks from backend code without importing the task implementation
Applied to files:
internal-packages/run-engine/src/engine/tests/trigger.test.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.ts
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Applies to apps/webapp/app/**/*.{ts,tsx} : Access all environment variables through the `env` export of `env.server.ts` instead of directly accessing `process.env` in the Trigger.dev webapp
Applied to files:
apps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-08-14T18:35:44.370Z
Learnt from: nicktrn
Repo: triggerdotdev/trigger.dev PR: 2390
File: apps/webapp/app/env.server.ts:764-765
Timestamp: 2025-08-14T18:35:44.370Z
Learning: The BoolEnv helper in apps/webapp/app/utils/boolEnv.ts uses z.preprocess with inconsistent default value types across the codebase - some usages pass boolean defaults (correct) while others pass string defaults (incorrect), leading to type confusion. The helper should enforce boolean-only defaults or have clearer documentation.
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-08-19T09:49:07.011Z
Learnt from: julienvanbeveren
Repo: triggerdotdev/trigger.dev PR: 2417
File: apps/webapp/app/routes/api.v1.projects.$projectRef.envvars.$slug.import.ts:56-61
Timestamp: 2025-08-19T09:49:07.011Z
Learning: In the Trigger.dev codebase, environment variables should default to `isSecret: false` when not explicitly marked as secrets in the syncEnvVars functionality. This is the intended behavior for both regular variables and parent variables.
Applied to files:
apps/webapp/app/env.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Scope idempotency keys globally or to current run using the scope parameter
Applied to files:
apps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `idempotencyKeys.create()` to create idempotency keys for preventing duplicate task executions
Applied to files:
apps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Export tasks with unique IDs within the project to enable proper task discovery and execution
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schedules.task()` for scheduled/cron tasks instead of regular `task()`
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.tsinternal-packages/run-engine/src/engine/systems/enqueueSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `schemaTask()` from `trigger.dev/sdk/v3` with Zod schema for payload validation
Applied to files:
apps/webapp/app/runEngine/services/triggerTask.server.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger.config.ts : Specify runtime environment (node or bun) in trigger.config.ts using the `runtime` property
Applied to files:
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts
📚 Learning: 2025-12-18T14:09:10.154Z
Learnt from: ericallam
Repo: triggerdotdev/trigger.dev PR: 2794
File: internal-packages/run-engine/src/engine/systems/debounceSystem.ts:390-397
Timestamp: 2025-12-18T14:09:10.154Z
Learning: In the debounce system (internal-packages/run-engine/src/engine/systems/debounceSystem.ts), millisecond delays are not supported. The minimum debounce delay is 1 second (1s). The parseNaturalLanguageDuration function supports w/d/hr/h/m/s units only.
Applied to files:
internal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Limit task duration using the `maxDuration` property (in seconds)
Applied to files:
internal-packages/run-engine/src/engine/systems/enqueueSystem.tsinternal-packages/run-engine/src/engine/systems/ttlSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use logger methods (debug, log, info, warn, error) from `trigger.dev/sdk/v3` for structured logging in tasks
Applied to files:
internal-packages/run-engine/src/engine/systems/enqueueSystem.ts
📚 Learning: 2025-11-27T16:27:35.304Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/writing-tasks.mdc:0-0
Timestamp: 2025-11-27T16:27:35.304Z
Learning: Applies to **/trigger/**/*.{ts,tsx,js,jsx} : Use `batch.triggerByTask()` to batch trigger tasks by passing task instances for static task sets
Applied to files:
apps/webapp/app/runEngine/types.tsapps/webapp/app/runEngine/services/batchTrigger.server.ts
📚 Learning: 2026-02-03T18:27:40.429Z
Learnt from: 0ski
Repo: triggerdotdev/trigger.dev PR: 2994
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.environment-variables/route.tsx:553-555
Timestamp: 2026-02-03T18:27:40.429Z
Learning: In apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.environment-variables/route.tsx, the menu buttons (e.g., Edit with PencilSquareIcon) in the TableCellMenu are intentionally icon-only with no text labels as a compact UI pattern. This is a deliberate design choice for this route; preserve the icon-only behavior for consistency in this file.
Applied to files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
📚 Learning: 2025-12-08T15:19:56.823Z
Learnt from: 0ski
Repo: triggerdotdev/trigger.dev PR: 2760
File: apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx:278-281
Timestamp: 2025-12-08T15:19:56.823Z
Learning: In apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx, the tableState search parameter uses intentional double-encoding: the parameter value contains a URL-encoded URLSearchParams string, so decodeURIComponent(value("tableState") ?? "") is required to fully decode it before parsing with new URLSearchParams(). This pattern allows bundling multiple filter/pagination params as a single search parameter.
Applied to files:
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx
📚 Learning: 2025-11-27T16:26:58.661Z
Learnt from: CR
Repo: triggerdotdev/trigger.dev PR: 0
File: .cursor/rules/webapp.mdc:0-0
Timestamp: 2025-11-27T16:26:58.661Z
Learning: Use the Run Engine 2.0 from `internal/run-engine` for new run lifecycle code in the webapp instead of the legacy run engine
Applied to files:
internal-packages/run-engine/src/engine/index.ts
🧬 Code graph analysis (15)
apps/webapp/app/v3/utils/queueLimits.server.ts (1)
apps/webapp/app/env.server.ts (1)
env(1332-1332)
internal-packages/run-engine/src/engine/systems/waitpointSystem.ts (1)
internal-packages/run-engine/src/engine/statuses.ts (1)
isFinalRunStatus(55-57)
apps/webapp/app/v3/runEngine.server.ts (2)
apps/webapp/app/env.server.ts (1)
env(1332-1332)apps/supervisor/src/env.ts (1)
env(144-144)
internal-packages/run-engine/src/run-queue/keyProducer.ts (1)
internal-packages/run-engine/src/run-queue/index.ts (4)
shard(1170-1236)shard(1258-1318)shard(1324-1350)shard(1420-1545)
apps/webapp/app/env.server.ts (2)
apps/webapp/app/utils/boolEnv.ts (1)
BoolEnv(12-14)apps/supervisor/src/envUtil.ts (1)
BoolEnv(15-17)
apps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts (1)
packages/core/src/v3/isomorphic/friendlyId.ts (1)
RunId(93-93)
apps/webapp/app/runEngine/services/triggerTask.server.ts (3)
internal-packages/run-engine/src/engine/errors.ts (1)
ServiceValidationError(69-78)apps/webapp/app/v3/services/common.server.ts (1)
ServiceValidationError(1-6)apps/webapp/app/utils/packets.ts (1)
handleMetadataPacket(11-37)
apps/webapp/app/presenters/v3/LimitsPresenter.server.ts (2)
apps/webapp/app/v3/runEngine.server.ts (1)
engine(11-11)apps/webapp/app/v3/utils/queueLimits.server.ts (2)
getQueueSizeLimit(24-33)getQueueSizeLimitSource(42-51)
internal-packages/run-engine/src/batch-queue/index.ts (3)
apps/webapp/app/runEngine/services/batchTrigger.server.ts (1)
batchId(164-350)packages/core/src/v3/apiClient/index.ts (1)
batchId(419-537)internal-packages/run-engine/src/engine/systems/batchSystem.ts (1)
batchId(39-137)
internal-packages/run-engine/src/engine/systems/enqueueSystem.ts (1)
packages/core/src/v3/isomorphic/duration.ts (1)
parseNaturalLanguageDuration(76-79)
apps/webapp/app/runEngine/types.ts (2)
internal-packages/run-engine/src/run-queue/fairQueueSelectionStrategy.ts (4)
env(501-521)env(560-578)env(580-594)env(596-616)internal-packages/run-engine/src/shared/index.ts (1)
AuthenticatedEnvironment(4-6)
apps/webapp/app/runEngine/concerns/queues.server.ts (2)
apps/webapp/app/v3/models/workerDeployment.server.ts (1)
findCurrentWorkerFromEnvironment(198-224)apps/webapp/app/runEngine/types.ts (1)
QueueValidationResult(38-46)
internal-packages/run-engine/src/run-queue/index.ts (1)
internal-packages/redis/src/index.ts (2)
Callback(4-4)Result(4-4)
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.limits/route.tsx (1)
apps/webapp/app/utils/numberFormatter.ts (1)
formatNumber(15-22)
internal-packages/run-engine/src/engine/index.ts (2)
internal-packages/run-engine/src/run-queue/index.ts (2)
options(1152-1168)options(1240-1256)internal-packages/run-engine/src/shared/index.ts (1)
MinimalAuthenticatedEnvironment(8-19)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (27)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: sdk-compat / Cloudflare Workers
- GitHub Check: typecheck / typecheck
- GitHub Check: sdk-compat / Deno Runtime
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: sdk-compat / Node.js 20.20 (ubuntu-latest)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: sdk-compat / Bun Runtime
- GitHub Check: sdk-compat / Node.js 22.12 (ubuntu-latest)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
internal-packages/run-engine/src/batch-queue/completionTracker.ts
Outdated
Show resolved
Hide resolved
ericallam
changed the title
ericallam
force-pushed
the
ea-branch-117
branch
from
aaea8d6 to
21dae6f
Compare
There was a problem hiding this comment.
Actionable comments posted: 8
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
internal-packages/run-engine/src/batch-queue/index.ts (1)
958-976:⚠️ Potential issue | 🟠 MajorCompletion check at Line 969 uses
meta.runCountwhich may be stale.
metais fetched once per message at Line 762-764. IfupdateRunCountwas called between the meta fetch and the completion check, the comparisonprocessedCount === meta.runCountcould use an outdated value, potentially missing or double-triggering finalization. For correctness, consider re-fetchingrunCountatomically alongside theprocessedCountincrement, or havingupdateRunCounttrigger a finalization check.
🤖 Fix all issues with AI agents
In
`@apps/webapp/app/routes/_app.orgs`.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx:
- Around line 367-370: The call to getQueueUsageColorClass(environment.queued,
environment.queueSizeLimit) is unresolved — add a definition or import for
getQueueUsageColorClass that mirrors the logic of the existing
getUsageColorClass used in the limits route: implement a function
getQueueUsageColorClass(queued: number | undefined, queueSizeLimit: number |
undefined) that returns a CSS class string (or null) based on usage percentage
thresholds and handles undefined values; place it in the same file (or import
it) so valueClassName can call it, and ensure it returns null when no class
should be applied so the existing env.paused fallback ("text-warning
tabular-nums" or "tabular-nums") continues to work.
- Around line 515-518: The per-queue limit check (isAtQueueLimit) is using
queue.queued against environment.queueSizeLimit which is an environment-wide
cap; change the comparison to use environment.queued (total queued across all
queues) versus environment.queueSizeLimit so the badge reflects environment
capacity (or implement a true per-queue limit if intended). Update the
isAtQueueLimit logic to reference environment.queued and
environment.queueSizeLimit (consistent with the BigNumber usage) and remove or
revise any reliance on queue.queued for this environment-wide badge.
In `@apps/webapp/app/v3/runEngineHandlers.server.ts`:
- Around line 722-754: The duplicated pre-failed run creation via
triggerFailedTaskService.call when isFinalAttempt can itself throw and cause a
second attempt in the outer catch — wrap each inner invocation of
triggerFailedTaskService.call (the calls inside the isFinalAttempt branch and
the analogous call in the outer catch) in a small try/catch so any errors from
that call are caught and handled locally (log the error to the existing
span/process logger and avoid rethrowing), ensure span.end() still runs, and
return the same successful pre-failed-run response when a failedRunId is
returned; this prevents a partial success from causing a duplicate pre-failed
run while preserving the current control flow for both the isFinalAttempt path
and the outer catch path.
In `@internal-packages/run-engine/src/batch-queue/completionTracker.ts`:
- Around line 116-130: The updateRunCount function performs a non-atomic
read-modify-write (getMeta → modify → storeMeta) which can lose updates under
concurrent callers; change it to perform the increment atomically in the backing
store (e.g., use a Redis Lua script or the same atomic pattern used by
recordSuccessIdempotent) so runCount is updated with a single atomic operation,
or enforce/document single-writer semantics; specifically replace the current
getMeta/storeMeta sequence in updateRunCount with an atomic increment/update
that targets the BatchMeta.runCount field (or wrap the logic in a Lua script
that reads, increments/sets and returns the new value) and update logging to
reflect the atomic result.
In `@internal-packages/run-engine/src/engine/index.ts`:
- Around line 2275-2300: The method `#resolveMaxTtl` currently returns the
original unparseable ttl string when parseNaturalLanguageDurationInMs(ttl)
yields undefined, which allows invalid TTLs to bypass the max clamp; change the
logic so that if ttl fails to parse but the configured options.defaultMaxTtl
parses successfully, you return the parsed max (i.e., maxTtl) to enforce the
cap; keep the existing fallback that if defaultMaxTtl itself is unparseable
(parseNaturalLanguageDurationInMs(maxTtl) === undefined) you return ttl as
before.
- Around line 2245-2273: The `#ttlExpiredCallback` currently calls
this.ttlSystem.expireRunsBatch after a Lua script has already removed runs from
Redis, so if expireRunsBatch throws the DB update fails the runs become
orphaned; to fix, add compensation in `#ttlExpiredCallback` to detect errors from
ttlSystem.expireRunsBatch and, on failure, re-add the affected runIds back into
the TTL structures (sorted set/queue/concurrency tracking) via a new ttlSystem
method (e.g., ttlSystem.requeueRuns or ttlSystem.restoreRuns) or invoke existing
requeue logic, and/or schedule the runIds for a background cleanup retry; ensure
the added code references `#ttlExpiredCallback`, ttlSystem.expireRunsBatch, and
the Lua removal behavior so the requeue/cleanup is called only when
expireRunsBatch fails.
In `@internal-packages/run-engine/src/engine/systems/ttlSystem.ts`:
- Around line 248-291: The batch update (taskRun.updateMany) followed by per-run
updates (taskRun.update and taskRunExecutionSnapshot.create) can leave runs
marked EXPIRED without snapshots/errors if any per-run op fails; change
ttlExpiredCallback to perform each run's status change, error write, and
snapshot creation inside a single per-run prisma.$transaction() (instead of the
global updateMany + Promise.all) so that setting status to EXPIRED,
completedAt/expiredAt, writing the error, and creating taskRunExecutionSnapshot
happen atomically for each run (operate on runsToExpire entries and replace uses
of taskRun.updateMany, taskRun.update, and taskRunExecutionSnapshot.create
accordingly), and ensure transaction failures are surfaced/logged so those runs
remain PENDING for the next TTL poll.
In `@internal-packages/run-engine/src/run-queue/index.ts`:
- Around line 2710-2748: The dequeue Lua branch that handles TTL expiration
(checks using ttlExpiresAt <= currentTime) currently removes the message from
queueKey, envQueueKey, messageKey and ttlQueueKey but never reports the expired
run to the application; to fix this, stop evicting TTL-expired messages in the
dequeue path: modify the ttlExpiresAt <= currentTime branch so it does not
remove the message or its TTL membership (i.e., do not call ZREM on
queueKey/envQueueKey or DEL messageKey or ZREM ttlQueueKey) and simply skip
adding the run to results (leaving the message for the TTL consumer to handle),
ensuring that results, dequeuedCount and SADD calls are untouched for the
non-expired branch; locate and change the code around the symbols ttlExpiresAt,
currentTime, queueKey, envQueueKey, ttlQueueKey, messageKey, results and
dequeuedCount.
🧹 Nitpick comments (10)
apps/webapp/app/env.server.ts (1)
536-540:QUEUE_SIZE_CACHE_ENABLEDuses numeric type instead ofBoolEnv.Other boolean-like env vars in this file (e.g.,
RUN_ENGINE_TTL_SYSTEM_DISABLED,BATCH_TRIGGER_CACHED_RUNS_CHECK_ENABLED) useBoolEnv. Usingz.coerce.number().int()for an enable/disable flag is inconsistent and forces consumers to compare against1instead of using a proper boolean.Suggested change
- QUEUE_SIZE_CACHE_ENABLED: z.coerce.number().int().optional().default(1), + QUEUE_SIZE_CACHE_ENABLED: BoolEnv.default(true),apps/webapp/app/runEngine/concerns/idempotencyKeys.server.ts (1)
120-122: Non-null assertion onassociatedWaitpoint!is safe but could be cleaner.The
!is safe because of the early return on line 98-100, but TypeScript can't narrowletvariables inside closures. Consider assigning to aconstafter the guard to eliminate the assertion.Suggested approach
if (!associatedWaitpoint) { return { isCached: true, run: existingRun }; } + const waitpoint = associatedWaitpoint; await this.traceEventConcern.traceIdempotentRun( ... await this.engine.blockRunWithWaitpoint({ runId: RunId.fromFriendlyId(parentRunId), - waitpoints: associatedWaitpoint!.id, + waitpoints: waitpoint.id,internal-packages/run-engine/src/engine/tests/getSnapshotsSince.test.ts (1)
214-218: Relaxed assertion removes verification of waitpoint completion in snapshots.The original assertion that
latest.completedWaitpoints.length > 0was removed and replaced with comments. While timing sensitivity is real, this means the test no longer verifies the core "waitpoints only on latest" behavior — it only checks that earlier snapshots have empty waitpoints.Consider adding a retry/poll loop (e.g., poll
getSnapshotsSinceuntil the latest snapshot hascompletedWaitpoints.length > 0or a timeout), so the test still validates the positive case without being flaky.internal-packages/run-engine/src/engine/systems/waitpointSystem.ts (1)
17-17: Derive/validate env + project from the run to prevent mismatches.
Nice use ofisFinalRunStatusto avoid creating waitpoints for terminal runs. Since you already load the run, consider using/validating itsprojectIdandruntimeEnvironmentIdinstead of trusting caller inputs, to avoid accidental cross‑environment linkage.🔧 Suggested adjustment
- const run = await this.$.prisma.taskRun.findFirst({ - where: { id: runId }, - include: { associatedWaitpoint: true }, - }); + const run = await this.$.prisma.taskRun.findFirst({ + where: { id: runId }, + select: { + status: true, + projectId: true, + runtimeEnvironmentId: true, + associatedWaitpoint: true, + }, + }); ... - const runAfterLock = await this.$.prisma.taskRun.findFirst({ - where: { id: runId }, - include: { associatedWaitpoint: true }, - }); + const runAfterLock = await this.$.prisma.taskRun.findFirst({ + where: { id: runId }, + select: { + status: true, + projectId: true, + runtimeEnvironmentId: true, + associatedWaitpoint: true, + }, + }); + if ( + runAfterLock.projectId !== projectId || + runAfterLock.runtimeEnvironmentId !== environmentId + ) { + throw new Error( + "getOrCreateRunWaitpoint: run does not match provided project/environment" + ); + } - const waitpointData = this.buildRunAssociatedWaitpoint({ projectId, environmentId }); + const waitpointData = this.buildRunAssociatedWaitpoint({ + projectId: runAfterLock.projectId, + environmentId: runAfterLock.runtimeEnvironmentId, + });Also applies to: 819-875
apps/webapp/app/runEngine/types.ts (1)
59-77: Prefer a type alias forQueueManager(repo guideline).
Since this interface is being updated, consider converting it to a type alias to match the TS style guide.As per coding guidelines "Use types over interfaces for TypeScript".♻️ Suggested refactor
-export interface QueueManager { +export type QueueManager = { resolveQueueProperties( request: TriggerTaskRequest, lockedBackgroundWorker?: LockedBackgroundWorker ): Promise<QueueProperties>; getQueueName(request: TriggerTaskRequest): Promise<string>; validateQueueLimits( env: AuthenticatedEnvironment, queueName: string, itemsToAdd?: number ): Promise<QueueValidationResult>; resolveQueueNamesForBatchItems( env: AuthenticatedEnvironment, items: Array<{ task: string; options?: { queue?: { name?: string } } }> ): Promise<Map<string, number>>; validateMultipleQueueLimits( env: AuthenticatedEnvironment, queueCounts: Map<string, number> ): Promise<{ ok: true } | { ok: false; queueName: string; maximumSize: number; queueSize: number }>; getWorkerQueue( env: AuthenticatedEnvironment, regionOverride?: string ): Promise<string | undefined>; -} +};apps/webapp/app/v3/runEngineHandlers.server.ts (1)
651-670: TriggerFailedTaskService is instantiated on every batch item invocation.The service is only used on final-attempt failures, but it's constructed eagerly at the top of every callback invocation. Consider moving the instantiation inside the
isFinalAttemptguards to avoid unnecessary allocations on the happy path.apps/webapp/app/runEngine/concerns/queues.server.ts (1)
421-436:getCachedQueueSizefails open on cache errors — verify this is intentional.Line 435 returns
result.val ?? 0, meaning if the cache fetch fails (returningundefined), the queue size is treated as 0. This allows requests through even when queue size can't be determined, which is a permissive-failure strategy. This is likely the right trade-off (availability over strict enforcement), but worth documenting.internal-packages/run-engine/src/run-queue/index.ts (3)
108-110: Usetypeinstead ofinterfaceforTtlSystemCallback.Per coding guidelines, prefer types over interfaces in TypeScript files.
Proposed fix
-export interface TtlSystemCallback { - (runs: Array<{ queueKey: string; runId: string; orgId: string }>): Promise<void>; -} +export type TtlSystemCallback = ( + runs: Array<{ queueKey: string; runId: string; orgId: string }> +) => Promise<void>;As per coding guidelines:
**/*.{ts,tsx}: Use types over interfaces for TypeScript.
669-677: TTL member uses|as delimiter — consider a safer encoding.The TTL member format
${queueKey}|${runId}|${orgId}relies on|never appearing in any of these values. While current ID formats make this safe today, a future change to key formats could silently break the parsing in theexpireTtlRunsLua script (Lines 2582–2588). A null byte (\0) or a fixed-width encoding would be more robust.
1713-1721: Minor: TTL shard is computed even when TTL system is disabled.The TTL shard derivation (Lines 1714–1718) runs unconditionally, though the result is only meaningful when
this.options.ttlSystemis set. This is functionally harmless since the empty-stringttlQueueKeydisables TTL logic in the Lua script, but wrapping it in the conditional would be slightly cleaner.
...app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx
Outdated
Show resolved
Hide resolved
...app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.queues/route.tsx
Show resolved
Hide resolved
internal-packages/run-engine/src/batch-queue/completionTracker.ts
Outdated
Show resolved
Hide resolved
…env queue size check
…ndles failures from queue length limit failures and also retries
… var, set at engine level
…properly cleaned up from queues and queues rebalanced after getting expired by ttl system
…not parse correctly
…e a default value of 500
…from the ttl system
ericallam
force-pushed
the
ea-branch-117
branch
from
94e138b to
d424754
Compare
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
…xisting run is already complete
| * Resolves queue names for batch items and groups them by queue. | ||
| * Returns a map of queue name -> count of items going to that queue. | ||
| */ | ||
| async resolveQueueNamesForBatchItems( |
There was a problem hiding this comment.
We might not need this anymore
| * Validates queue limits for multiple queues at once. | ||
| * Returns the first queue that exceeds limits, or null if all are within limits. | ||
| */ | ||
| async validateMultipleQueueLimits( |
There was a problem hiding this comment.
same thing, might not be needed
|
|
||
| return batch; | ||
| } | ||
| case "ABORTED": { |
There was a problem hiding this comment.
Clean this up, no longer doing it this way in the new streaming batch stuff
1 participant
This PR implements a new run TTL system and queue size limits to prevent unbounded queue growth which should help prevent situations where queues enter a "death spiral" where the queue will never be able to catch up.
The main/correct way to battle this situation is to enforce a maximum TTL on all runs (e.g. up to 14 days) where runs that have been queued for that maximum TTL will get auto-expired, making room for newer runs to execute. This required creating a new TTL system that can handle higher workloads and is now deeply integrated into the RunQueue. When runs are enqueued with a TTL, they are added to their normal queue as well as to the TTL queue. When runs are dequeued, they are removed from both their normal queue and the TTL queue. If runs are dequeued by the TTL system, they are removed from their normal queue. Both these dequeues happen automatically so there is no race condition.
The TTL expiration system is also made reliable by expiring runs via a Redis worker, which is enqueued to atomically inside the TTL dequeue lua script.
Optional associated waitpoints
Additionally, this PR implements an optimization where runs that aren't triggered with a dependent parent run will no longer create an associated waitpoint. Associated waitpoints are then lazily created if a dependent run wants to wait for the child run post-facto (via debounce or idempotency), which is a rare situation but is possible. This means fewer waitpoint creations but also fewer waitpoint completions for runs with no dependencies.
Environment Queue Limits
Prevents any single queue growing too large by enforcing queue size limits at trigger time.
Batch trigger fixes
Currently when a batch item cannot be created for whatever reason (e.g. queue limits) the run will never get created, which means a stalled run if using
batchTriggerAndWait. We've updated the system to handle this differently: now when a batch item cannot be triggered and converted into a run, we will eventually (after retrying 8 times up to 30s) we will create a "pre-failed" run with the error details, correctly resolving the batchTriggerAndWait.